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We present an automated verification of the well-known modal logic cube in Isabelle/HOL, in which 
we prove the inclusion relations between the cube’s logics using automated reasoning tools. Prior 
work addresses this problem but without restriction to the modal logic cube, and using encodings in 
first-order logic in combination with first-order automated theorem provers. In contrast, our solution 
is more elegant, transparent and effective. It employs an embedding of quantified modal logic in 
classical higher-order logic. Automated reasoning tools, such as Sledgehammer with LEO-II, Satal- 
lax and CVC4, Metis and Nitpick, are employed to achieve full automation. Though successful, the 
experiments also motivate some technical improvements in the Isabelle/HOL tool. 


1 Introduction 

We present an approach to meta-reasoning about modal logics, and apply it to verify the relative strengths 
of logics in the well-known modal logic cube, which is illustrated in Figure 1. In particular, proofs are 
given for the equivalences of different axiomatizations and the inclusion relations shown in the cube. 
Our solution makes extensive use of the fact that all modal logics found in the cube are sound and 
complete because they arise from base modal logic K by adding Sahlqvist axioms. This is in contrast 
to prior work by Rabe et al. ifT^ . who address the more general problem of determining the relation 
between two arbitrary modal logics characterized by their sets of inference rules. In their article the 
authors apply first-order logic encodings in combination with first-order automated theorem provers to 
prove an inclusion relation employing a number of different decision strategies. For the subproblem 
of only comparing logics within the cube (and therefore taking advantage of normality as additional 
knowledge) our solution improves on the elegance and simplicity of the problem encodings, as well 
as with automation performance. One motivation of this paper is to demonstrate the advantage of a 
pragmatically more expressive logic environment (here classical higher-order logic) in comparison to a 
less expressive language such as first-order logic or decidable fragments thereof. 

We exploit an embedding of quantified multimodal logic (QML) in classical higher-order logic (HOL) 
fJl, in which we carry out the automated verification of the aforementioned inclusion relations. These 
include the logics K, D, M (also known as T), S4, and S5. We analyze inclusion and equivalence relations 
for modal logics that can be defined from normal modal logic K by adding (combinations of) the axioms 
M, B, D, 4, and 5. In our problem encodings we exploit the well-known correspondences between these 
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= M5 = MBS = M4B5 
= M45 = M4B - D4B 
= D4B5 = DBS 


M: 

□P- 


B: 


D: 

□p-s>0P 

4: 

□p- 


5: 

M 

OP ^ OOP 



= K4BS = K4B 


Figure 1: The modal logic cube: reasoning in modal logics is commonly done with respect to a certain 
set of basic axioms; different choices of basic axioms give rise to different modal logics. These modal 
logics can be arranged as vertices in a cube, such that the edges between them denote inclusion relations. 


axioms and semantic properties of accessibility relations (i.e. Kripke models). These correspondences 
can themselves be elegantly formalized and effectively automated in our approach. Formalization of 
the modal axioms M, B, D, 4, and 5 requires quantification over propositional variables. This explains 
why an embedding of quantified modal logic in HOL is needed here, and not simply an embedding of 
propositional modal logic in HOL. 

Our previous work (see the non-refereed, invited paper fSj]) has already demonstrated the feasibility 
of the approach. However, instead of the development done there in pure TPTP THF O, we here 
work with Isabelle/HOL lfT4l as the base environment, and fruitfully exploit various reasoning tools 
that are provided with it. This includes the Sledgehammer-based ifTSl interfaces from Isabelle/HOL 
to the external higher-order theorem provers LEO-II ||9l and Satallax l|Tl, as well as Isabelle/HOL’s 
own reasoner Metis lOTl . Moreover, the higher-order model finding capabilifies of Nifpick ifTOl are 
heavily used in order fo formulaic and prove subsequenf inclusion fheorems in Isabelle/HOL. We also 
encounfered some problems wifh inleracling wifh Ihe proof reconslruclion available for LEO-II and 
Salallax in Isabelle/HOE. 

This paper is a verified documenl in fhe sense lhaf if has been aulomafically generafed from Isabelle/HOE 
source code wifh fhe help of Isabelle’s build fool (fhe enlire source package is available from hffp: 
//chrisloph-benzmueller.de/varia/pxlp2015.zip). 

The paper is sfrucfured as follows: Seclion[^presenls an encoding of QME in HOE. This part reuses fhe 
theory provided by Benzmuller and Paulson which has recently been further developed (to cover full 
higher-order QME) and applied for the verification of Godel’s ontological argument Bini. Section 
first establishes the well-known correspondence between properties of models and base axioms, and 
then investigates the equivalence of different axiomatizations. Subsequently, all inclusion relations as 
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depicted in the modal logic cube are shown to be proper. Finally, the minimal number of possible worlds 
that is required to obtain proper inclusions in each case is determined and verified. Section [^presents a 
short evaluation and discussion of the conducted experiments, and Section [^concludes the paper. 

2 An Embedding of Quantified Multimodal Logics in HOL 

In contrast to the monomodal case, in quantified multimodal logics both modalities □ and 0 are parame¬ 
trized, such that they refer to potentially different accessibility relations. We write and 0^ to refer to 
necessity and possibility wrt. a relation R. Furthermore, in terms of quantification, we only consider the 
constant-domain case: this means that all possible worlds share one common domain of discourse. More 
details on the embedding of QML in HOL are given in earlier work 171161. 

QML formulas are translated as HOL terms of type i bool, where i is the type of possible worlds. This 
type is abbreviated as a. 

The classical connectives A, —and V (which quantifies over individuals and over sets of individuals) 
and 3 (over individuals) are lifted to type a. The lifted connectives are A'", V'”, ='”, V, and 

3 (the latter two are modeled as constant symbols). Other connectives can be introduced analogously. 
Moreover, the modal operators □ and 0, parametric to R, are introduced. Note that in symbols like -i"*, 
symbol m is simply part of the name, whereas in and 0^, symbol is a parameter to the modality. 

abbreviation mnot:: a ^ a where <p = (Xw. ^ <p w) 

abbreviation mand :: ci => ci => ci where (p A'" xj/ = {Xw. (p w A \j/ w) 

abbreviation mor :: a ^ a a where (p V'" y/ = (Xw. (p w V w) 

abbreviation mimplies :: a ^ a ^ a where (p -a’” y/ = (Xw. (p w —!> y/ w) 

abbreviation mequiv:: a cr a where (p y/ = (Aw. (p w <—>■ y/ w) 

abbreviation mforall w ('a =A a) ^ a where V <I> = (Aw. Vx. <I>x w) 

abbreviation mexists :: {'a ^ a) a where 3 <I> = (Aw. 3x. Ox w) 

abbreviation mbox :: (/ ^ i bool) a ^ a where (p= (Aw. Vv. (Rw v) — (p v) 

abbreviation mdia :: (i i => bool) a ^ a where 0^ <p = (Aw. 3v. R w v A <p v) 

For grounding lifted formulas, the meta-predicate [•], read valid, is introduced, 
abbreviation valid :: a ^ bool where \p\ = Vw. /? w 


3 Reasoning about Modal Logics 

3.1 Correspondence Results 

Axioms of the modal cube correspond to constraints on the underlying accessibility relations. These 
constraints are as follows: 


definition refl = XR :: (i => 
definition sym = XR :: (i - 
definition ser = XR :: (i => 
definition trans = XR :: (i 
definition eucl = XR :: (i = 


■bool).yS.RSS 

> bool).y ST. (RST — >RTS) 
bool).yS.3T.RST 

=y bool). ySTU. (RST ARTU 

> bool). ySTU.(RSTARSU- 


aRSU) 

RTU) 


— reflexivity 

— symmetty 

— seriality 

— transitivity 

— Euclidean 


The corresponding axioms are defined next; note that they are parametric over accessibility relation R\ 
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definition M = XR . valid (V {XP. (D^ P) P)) 
definition B = XR . valid (V {XP. P —J-™ □‘^0’^ P)) 
definition D = XR . valid (V {XP. (□‘^ P) —0^ P)) 
definition IV = XR . valid (V {XP. (D^ P) — P)) 
definition V = XR . valid (V {XP. {()^ P) —□^0'^ P)) 

We will see below that correspondence theorems (between axioms and constraints on accessibility re¬ 
lations) can be elegantly expressed in HOL by exploiting the embedding used above. These correspon¬ 
dence theorems link a constraint to every axiom—for instance, M is linked to refl. Subsequently, in order 
to make statements about the relationship of two logics in the cube, it is sufficient to only look at the 
model constraints of their respective axiomatizations. Throughout the rest of this paper, all reasoning 
will be done on the model-theoretic side and then interpreted on the proof-theoretic side by the means of 
this correspondence. 


3.1.1 Axiom M corresponds to Reflexivity 

theorem A7: (V7?. {reflR) < — > {MR)) by {metis M-def refl-def) 

3.1.2 Axiom B corresponds to Symmetry 

lemma A2-fl: {\/R. {symR) —!• {B R)) by {metis B-def sym-def) 
lemma A2-7>: (\/R. {B R) —>■ {sym R)) by {simp add:B-def sym-def, force) 
theorem A2: (\/R. {sym R) i — {B R)) by {metis A2-a A2-b) 

3.1.3 Axiom D corresponds to Seriality 

theorem A3: {VR. {ser R) < —^ {DR)) by {metis D-def ser-def) 

3.1.4 Axiom 4 corresponds to Transitivity 

theorem A4: (V/?. {trans R) <—{IV R)) by {metis IV-def trans-def) 


3.1.5 Axiom 5 corresponds to Euclideanness 

lemma A5-a: {VR. {euclR) — > {VR)) by {metis V-def eucl-def) 
lemma Ad-f?: (\/R. {VR) —>■ {eucl R)) by {simp add:V-def eucl-def, force) 
theorem A5: {\/R. {euclR) < —>■ {VR)) by {metis A5-a A5-b) 


3.2 Alternative Axiomatisations of Modal Logics 


Often the same logic within the cube can be obtained through different axiomatizations. In this section 
we show how to prove different axiomatizations for logic S5 resp. KBS to be equivalent. Using the 
correspondence theorems from the previous section, the equivalences can be elegantly formulated solely 
using the properties of accessibility relations. In Subsections 3.2.1 and 3.2.2| we also add the correspond¬ 
ing statements using the modal logic axioms; this could analogously be done also for the other theorems 
and lemmata presented in Sections 3.2 and 3.3. 


The theorems below can be solved directly by Metis when it is provided the minimal set of necessary def¬ 
initions. Sledgehammer (with the ATPs LEO-II and Satallax or with first-order provers) can also quickly 
solve these problems, in which case the manual selection of the required definitions is not necessary. 
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3.2.1 MS MBS 

theorem Bl: \/R.{{refl R) A {eucl R)) <—5- {{refl R) A {sym R) A {eucl R)) 
by {metis eucl-def refl-def sym-def) 

ihtortm B1-alt: \/R.{{MR) A {VR)) <—^ {{MR) A {B R) A {VR)) 
by {metis A1 A2 A5 Bl) 

3.2.2 MS M4BS 

theorem B2: \/R.{{refl R) A {eucl R)) < —s- {{refl R) A {trans R) A {sym R) A {eucl R)) 
by {metis eucl-def refl-def trans-def sym-def) 

theorem B2-alt:yR.{{MR) A {VR)) <—^ {{MR) A {IVR) A {B R) A {VR)) 
by {metis A1 A4A5 Bl-alt B2) 

3.2.3 MS M4S 

theorem B3-. yR.{{refl R) A {eucl R)) < —5- {{refl R) A {trans R) A {eucl R)) 
by {metis eucl-def refl-def trans-def) 

3.2.4 MS M4B 

theorem B4: \/R.{{refl R) A {eucl R)) i —s- {{refl R) A {trans R) A {sym R)) 
by {metis eucl-def refl-def sym-def trans-def) 

3.2.5 MS D4B 

theorem B5; yR.{{refl R) A {euclR)) < —5- {{ser R) A {trans R) A {symR)) 
by {metis eucl-def refl-def ser-def sym-def trans-def) 

3.2.6 MS D4BS 

theorem B6: yR.{{refl R) A {eucl R)) <—{{ser R) A {trans R) A {sym R) A {eucl R)) 
by {metis eucl-def refl-def ser-def sym-def trans-def) 

3.2.7 MS DBS 

theorems?: \/ R.{{refl R) A {eucl R)) i —S- {{ser R) A {sym R) A {eucl R)) 
by {metis eucl-def refl-def ser-def sym-def) 

3.2.8 KBS K4BS 

theorem B8: 'iR.{{sym R) A {eucl R)) <—> {{trans R) A {sym R) A {eucl R)) 
by {metis eucl-def sym-def trans-def) 

3.2.9 KBS K4B 

theorem B9: 'yR.{{sym R) A {eucl R)) <—> {{trans R) A {sym R)) 
by {metis eucl-def sym-def trans-def) 


3.3 Proper Inclusion Relations between Different Modal Logics 

An edge within the eube denotes an inelusion between the eonneeted logies. In the forward direetion, 
these ean be trivially shown valid through monotonieity of entailment and equivalenee of the different 
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axiomatizations. For example, for the forward link from logic K to logic B, we need to show that every 
theorem of K is also a theorem of B; this simply means to disregard the additional axiom B. Below, the 
crucial backward directions are proved. Informally, it is shown that through moving further up in the 
cube (adding further axioms), theorems can be proved which were not provable before; this means that 
the inclusions are proper. We write A > B to indicate that logic A can prove strictly more theorems than 
logic B. 

It has to be noted that some logics are actually equivalent if the only models considered have few enough 
worlds; examples are given below. We introduce some useful abbreviations to formulate constraints on 
the number of worlds in a model. 

abbreviation one-world-model i ^ bool where wl = Vx. x = wl 

abbreviation two-world-model i ^ i ^ bool where wl w2 = (Vx. x = wl M x = w2) f\wl ^w2 
abbreviation three-world-model:: i ^ i ^ i ^ bool where wl w2 w3 = (Vx. x = wl \l x = w2 M x = w3) A 
wl ^ w2 /\ wl ^ w3 /\ w2 ^ w3 

In what follows, we reserve the symbols il, i2 and i3 for worlds, and r for an accessibility relation. 

We applied the following methodology in the experiments reported in this section: 

(Step A) First we deliberately made invalid conjectures about inclusion relations—e.g. for proving 
K4 > K we first wrongly conjectured that K4 C K, meaning that K4 entails K. We did this by 
conjecturing 

lemma Cl-A: \/R. {trans R) 

These wrongly-conjectured lemmata in Step A are uniformly named C*-A. Note that for the for¬ 
mulation of the C*-A-lemmata we again exploit the correspondence results given earlier, and we 
work with conditions on the accessibility relations instead of using the corresponding modal logic 
axioms. For each C*-A-lemma Nitpick quickly generates a countermodel, which it communicates 
in a specific syntax. For example, the countermodel it presents for Cl-A is 

R = (Ax. -)(/i := (Ax. -)(/i := True, i 2 ■= True), (2 := (Ax. -)(/i := True, (2 := False)) . 

Diagrammatically this 2-world countermodel can be represented as follows 



(Step B) Next, we systematically employed the arity information obtained from the countermodels for 
the C*-A-lemmata, reported by Nitpick, to formulate a corresponding lemma to be passed via 
Sledgehammer to the HOL-ATPs LEO-II, Satallax and/or CVC4 O (whenever it was not trivially 
provable by the automation tools simp, force and/or blast available within Isabelle/HOL). In our 
running example this lemma is 

Cl-B: il 12 — 7- V/?. -1 trans R 

All but four of these lemmata can actually be proved by either LEO-II or Satallax. Some of the 
easier problems can already be automated with simp, force and blast, which are preferred here. 
The four cases in which no automation attempts succeeded (we also tried all other integrated 
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ATPs in Isabelle) are named C*-ATP-challenge below. Moreover, there are ten problems named 
C*-Isabelle-challenge. For these problems LEO-II or Satallax found proofs, but their Metis-based 
integration into Isabelle failed. Hence, no verification was obtained for these problems. However, 
we found that five of these C*-Isabelle-challenge problems can also be proved by CVC4, for which 
proof integration worked. Unfortunately, no other automation means (including the integrated first- 
order ATPs or SMT solvers) succeeded for the C*-Isabelle-challenge problems. 

(Step C) For the verification of the modal logic cube, the non-proved or non-integrated C*-challenge 
problems of Step B are clearly unsatisfactory, since no proper verification in Isabelle is obtained. 
However, an easy solution for these (and all other) cases is possible by exploiting not only Nitpick’s 
arity information on the countermodels, but by using all the information about the countermodels 
it presents, that is, the precise information on the accessibility relation. For example, Nitpick’s 
countermodel for Cl-A from above can be converted into the following theorem (where r denotes 
a fixed accessibility relation) 

theorem Cl-C: il i2 A r il il A r il i2 A r i2 il A —'r i2 i2 —> trans r. 

The resulting theorems we generate are uniformly named C*-C. It turns out that all C=t:-C-theorems 
can be quickly verified in Isabelle by Metis. Thus, for each link in the modal logic we provide 
either a verified C*-B theorem or, if this was not successful, a verified C*-C theorem. Taken 
together, this confirms that the inclusion relation in the cube are indeed proper. Hence, these C*-B 
resp. C*-C theorems complete the verification of the modal logic cube. Below the C*-C proof 
attempts are omitted if the corresponding C*-B attempts were already successful. 

(Step D) We additionally prove that the countermodels found by Nitpick in Step A are minimal (regard¬ 
ing the number of possible worlds). In other words, we prove here that the world model constraints 
as exploited in Step B are in fact minimal constraints under which the inclusion relations can be 
shown to be proper. Of course, if such a countermodel consists of one possible world only, nothing 
needs to be shown. 

Note that the entire process sketched above, that is the schematic Steps A-D, could be fully automated, 
meaning that the formulation of the lemmata and theorems in each step could be obtained automatically 
by analyzing and converting Nitpick’s output. In our experiments we still wrote and invoked the veri¬ 
fication of each link in the modal cube manually however. Clearly, automation facilities could be very 
useful for the exploration of the meta-theory of other logics, for example, conditional logics |41, since 
the overall methodology is obviously transferable to other logics of interest. 


3.3.1 K4 > K 

lemma Cl-A: \/R. trans R nitpick oops 

theorem Cl-B: il i2 — > (VI?, trans R) by {simp add:trans-def, force) 
lemma Cl-D: #' il —> iflR. trans R) by {metis {lifting^ full-types) trans-def) 



3.3.2 K5 > K 

lemma C2-A: WR. eucl R nitpick oops 

theorem C2-B: il i2 —>■ ^ (flR. eucl R) by {simp add:eucl-def, force) 
lemma C2-D: #' il —> {VR. eucl R) by {metis {lifting, full-types) eucl-def) 
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3.3.3 KB > K 

lemma C3-A: WR. sym R nitpick oops 

theorem C3-B: il i2 —^ (V/?. sym R) by {simp add:sym-def, force) 
lemma C3-D: #* il — if/R. sym R) by {metis {full-types) sym-def) 

3.3.4 K45 > K4 

lemma C4-A: WR. ser R —> {ser R A eucl R) nitpick oops 

lemma C4-B-Isabelle-challenge: il i2 —> ^ (\/R. ser R —> {ser R A eucl R)) 

— sledgehammer [remote Jeo2] (ser_def eucLdef) 

— CPU time: 13.74 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] - timed out oops 

theorem C4-C: il i2 A il il A r il i2 A r i2 il A i2 i2 — ^ {ser r — {ser r A 
by {metis ser-def eucl-def) 

lemma C4-D: #' il —> {WR. ser R —^ {serR A euclR)) by {metis {full-types) eucl-def) 


3.3.5 K45 > K5 

lemma C5-A: WR. eucl R —^ {ser R A eucl R) 

nitpick oops 

lemma C5-B-Isabelle-challenge\ il —> ^ (f/R. {eucl R) — > {ser R) A {eucl R)) 

— sledgehammer [remote Jeo2](eucLdef ser_def) - CPU time: 14.61 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] - timed out oops 

theorem C5-C: #* il A il il —> ^ {eucl r —> {ser r A eucl r)) by {metis (full-types) eucl-def ser-def) 





3.3.6 KBS > KB 


{sym R A eucl R) 



lemma C6-A: WR. sym R 

nitpick oops 

lemma C6-B-Isabelle-challenge: il i2 —> ^ (\/R. sym R —^ {sym R A eucl R)) 

— sledgehammer [remote Jeo2,timeout=200](sym_def eucLdef) - CPU time: 29.0 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] suggested following line: 
by {metis {full-types) A4 B8 Cl-B IV-def sym-def) 

lemma C6-D: #' il — (f/R. sym R —(sym R A eucl R)) 
by {metis (full-types) eucl-def) 



3.3.7 KBS > K45 

lemma C7-A: \/R. ser R A eucl R —{sym R A eucl R) 

nitpick oops 

lemma C7-B-Isabelle-challenge: if il i2 —^ (f/R. ser R A eucl R —> {sym R A eucl R)) 

— sledgehammer [remoteJeo2] (ser_def eucLdef sym_def) - CPU time: 11.15 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] - timed out oops 

theorem C7-C: if il i2 A r il il A ^ r il i2 A r i2 il A ^ r i2 i2 —> ^ {ser r A eucl r —> {sym r A eucl r)) 
by {metis full-types) ser-def eucl-def sym-def) 

lemma C7-D: #* il — > (f/R. ser R A eucl R —>■ {sym R A eucl R)) by {metis full-types) sym-def) 



12 
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3.3.8 D > K 

lemma C8-A: WR. serR nitpick oops 

lemma C8-B: #* il —> ^{WR. {ser R)) by {simp add:ser-def, force) 
theorem C8-C: #* il A il il —^{ser r) by {metis {full-types) ser-def) 



3.3.9 D4 > K4 

lemma C9-A: WR. trans R — s- {ser R A trans R) 

nitpick oops 

theorem C9-B: #* il — 5- ^ (flR. trans R — {ser R A trans R)) 
using Cl-D C8-B by blast 



3.3.10 D5 > K5 

lemma CIO-A: WR. eucl R — s- {ser R A eucl R) nitpick oops 

theorem CIO-B: #* il —^ (f/R. eucl R —^ {ser R A eucl R)) using B9 C3-D C9-B by blast 



3.3.11 D45 > K45 

lemma Cll-A: WR. trans R A eucl R —^ {ser R A trans R A eucl R) 

nitpick oops 

theorem Cll-B: #* il —> {\/R. trans R A eucl R —^ {ser R A trans R A eucl R)) 

using B9 C3-D C9-B by blast 




3.3.12 DB > KB 

lemma C12-A: \/R. sym R —^ {ser R A sym R) 

nitpick oops 

theorem C12-B: #* il —>■ ^ (f/R. sym R —> {ser R A sym R)) 
using Cll-B C3-D by blast 



3.3.13 S5 > KBS 


lemma C13-A: V/?. sym R A eucl R — 5- {refl R A eucl R) 

nitpick oops 

theorem C13-B: #* il —^ (f/R. sym R A eucl R — s- {refl R A eucl R)) using B5 C12-B C6-D by blast 



3.3.14 D4 > D 


{ser R) A {trans R) 


lemma C14-A: WR. {ser R) 

nitpick oops 

theorem C14-B-Isabelle-challenge: il i2 —> -^{\/R. ser R — 5- {ser R A trans R)) 

— sledgehammer [remoteJeo2] (ser.def trans.def) - CPU time: 13.08 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] suggested following line: 
by {metis {full-types) Cl-B trans-def ser-def) 

lemma C14-D: il —> {WR. serR —{ser R A trans R)) by {metis {full-types) trans-def) 
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3.3.15 D5 > D 

lemma C15-A: \/R. ser R —)■ {ser R A eucl R) 

nitpick oops 

theorem C15-B-Isabelle-challenge: il i2 —^ (V/?. ser R —> {ser R A eucl R)) 

— sledgehammer [remote Jeo2] (ser_def eucLdef) 

— CPU time: 12.9 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] suggested following line: 
by {metis {full-types) C14-B-Isabelle-challenge trans-def eucl-def) 
lemma C15-D: il —> (V/?. serR —> {ser R A eucl R)) by {metis {full-types) C2-D) 

3.3.16 DB > D 

lemma C16-A: \/R. ser R —{ser R A sym R) 

nitpick oops 

lemma C16-B: il i2 —> ^ (f/R. ser R —{ser R A 
lemma C16-D: il —s- (flR. ser R —^ {ser R A sym 

3.3.17 D45 > D4 

lemma C17-A: WR. ser R A trans R —{ser R A trans 

nitpick oops 

lemma C17-B-ATP-challenge: il i2 —> ^(fJR. ser R A trans R —)• {ser R A trans R A eucl R)) 
oops — All ATPs time out 

theorem C17-C: il i2 A r il il A r il i2 A ^ r i2 il A r i2 i2 —> ^ {ser r A trans r —> {ser r A trans r A eucl 

r)) 

by {metis {full-types) ser-def trans-def eucl-def) 

lemma C17-D: il —)■ (V/?. ser R A trans R —{ser R A trans R A eucl R)) 
by {metis {full-types) eucl-def) 

3.3.18 D45 > D5 

lemma C18-A: \/R. ser R A eucl R —{ser R A trans R A eucl R) 

nitpick oops 

lemma C18-ATP-challenge: if il i2 i3 —> ^ (iR. ser R A eucl R —{ser R A trans R A eucl R) ) 
oops — All ATPs time out 

theorem C18-C: if il i2 i3 A r il il A r il i2 A ^ r il i3 A r i2 il A r i2 i2 A^r i2 i3 A ^ r i3 il A r i3 i2 A ^ r 
i3 i3 —> {ser r A eucl r —> {ser r A trans r A eucl r)) by {metis full-types) eucl-def ser-def trans-def) 
lemma C18-D: if il i2 —> If/R. ser R A eucl R —{ser R A trans R A eucl R)) 
by {metis full-types) eucl-def trans-def) 





sym R)) by {simp add:ser-def sym-def, force) 
/?)) by {metis full-types) sym-def) 


R A eucl R) 





3.3.19 M > D 

lemma C19-A: WR. ser R —refl R 

nitpick oops 
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theorem C19-B-Isabelle-challenge: il i2 —^ {\/R. ser R —> reflR) 

— sledgehammer [remote Jeo2,timeout=200] (ser_def refl_def) - CPU time: 29.15 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] suggested following line: 
by {metis (full-types) C14-B-Isabelle-challenge trans-def refl-def) 

lemma C19-D: il —> (f/R. serR — reflR) by (metis (full-types) ser-def refl-def) 


3.3.20 S4>D4 

lemma C20-A: \fR. ser R A trans R —(refl R A trans R) 

nitpick oops 



lemma C20-B-Isabelle-challenge: il i2 —> ^ If/R. ser R A trans R —(refl R A trans R)) 

— sledgehammer [remote Jeo2](ser_def trans_def refl_def) - CPU time: 12.5 s. Metis reconstruction failed. 

— sledgehammer [cvc4,timeout=300] - timed out 

oops 

theorem C20-C: il i2 A r il il A ^ r il i2 A r i2 il A ^ r i2 i2 —> ^ (ser r A trans r —> (refl r A trans r)) 
by (metis (full-types) ser-def refl-def trans-def) 
lemma C20-D: il —> (WR. ser R A trans R —> (refl R A trans R)) 
by (metis (full-types) ser-def refl-def) 


3.3.21 S5 > D45 

lemma C21-A: \/R. ser R A trans R A eucl R —(refl R A eucl R) 

nitpick oops 



lemma C21 -B-lsabelle-challenge: il i2 — > ^ l\/R. ser R A trans R A eucl R —(refl R A eucl R)) 

— sledgehammer [remote Jeo2](ser_def trans_def eucLdef refl_def) - CPU time: 12.51 s. Metis reconstruction 
failed. 

— sledgehammer [cvc4,timeout=300] - timed out 

oops 

theorem C21-C: il i2 A r il il A ^ r il i2 A r i2 il A ^ r i2 i2 —^ (ser r A trans r A eucl r —(refl r A 
eucl r)) 

by (metis (full-types) ser-def trans-def eucl-def refl-def) 

lemma C21 -inclusion: #* il — (f/R. ser R A trans R A eucl R —(refl R A eucl R)) 
by (metis (full-types) ser-def refl-def) 


3.3.22 B > DB 

lemma C22-A: WR. ser R A sym R — (refl R A sym R) 

nitpick oops 

lemma C22-B-lsabelle-challenge: il i2 — > ^ If/R. ser R A sym R — (refl R A sym R)) 

— sledgehammer [remote_leo2,timeout=200](ser_def sym_def refl_def) - CPU time: 31.18 s. Metis reconstruction 
failed. 

— sledgehammer [cvc4,timeout=300] suggested following line: 

— by (smt C14_B sym_def trans_def refl_def) oops 

theorem C22-C: il i2 A r il il A r il i2 A r i2 il A ^ r i2 i2 —> ^ (ser r A sym r —(refl r A sym r)) 
by (metis (full-types) ser-def sym-def refl-def) 
lemma C22-D: il —> (f/R. ser R A sym R — (refl R A sym R)) 
by (metis (full-types) ser-def refl-def) 
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3.3.23 B > M 

lemma C23-A: \/R. refl R —!• {refl R A sym R) nitpick oops 
lemma C23-B-ATP-challenge: il i2 —>■ ^ (WR. refl R —>■ {refl R A sym R)) 
oops — All ATPs time out 

theorem C23-C: il i2 A r il il A r il i2 A ^ r i2 il A r i2 i2 —> ^ {refl r —>■ {refl r A sym r)) 
by {metis refl-def sym-def) 

lemma C23-D: il —> (\/R. refl R —{refl R A sym R)) by {metis {full-types) sym-def) 

3.3.24 S5>S4 

lemma C24-A: WR. refl R A trans R —{refl R A eucl R) 

nitpick oops 

lemma C24-B-ATP-challenge: il i2 —>■ ^ (f/R. refl R A trans R —> {refl R A eucl R)) 
oops — All ATPs time out 

theorem C24-C: il i2 A r il il A r il i2 A ^ r i2 il A r i2 i2 —> ^ {refl r A trans r —> {refl r A eucl r)) 
by {metis {full-types) trans-def refl-def eucl-def) 

lemma C24-D\ il — fiR. refl R A trans R —> {refl R A eucl R)) by {metis {full-types) eucl-def) 




3.3.25 S5 > B 

lemma C25-A: WR. refl R A sym R — >■ {refl R A eucl R) 

nitpick oops 

lemma C25-B-ATP-challenge: il i2 i3 — > ^ (f/R. {refl R A sym R) —> {refl R A eucl R)) 
oops — All ATPs time out 

theorem C25-C: il i2 i3 A r il il A r il i2 A ^ r il i3 A r i2 il A r i2 i2 A r i2 i3 A ^ r i3 il A r i3 i2 A r i3 i3 
—> {{refl r A sym r) —> {refl r A eucl r)) 
by {metis {full-types) eucl-def refl-def sym-def) 

lemma C25-D: il i2 —>■ tf/R. {refl R A sym R) —> {refl R A eucl R)) 
by {metis {full-types) refl-def sym-def eucl-def) 



4 Discussion and Future Work. 

The entire Isabelle document can be verified by Isabelle2014 in less than 60s on a semi-modern computer 
(2.4 GHz Core 2 Duo, 8 GB of memory). When including all (commented) remote calls to the external 
ATPs in the calculation the verification time sums up to a few minutes, which is still very reasonable. 
The improvements in comparison to the first-order based verification of the modal logic cube done earlier 
by Rabe et al. ifT^ . are: clarity and readability of the problem encodings, methodology, reliability (our 
proofs are verifiable in Isabelle/HOL) and, most importantly, automation performance. For the latter note 
that the experiments by Rabe et al. lIT^ required several days of reasoning time in first-order theorem 
provers. Most importantly, however, their solution relied on an enormous manual coding effort. However, 
we want to point again to the more general aims of their work. 

Our solution instead requires a small amount of resources in comparison. In fact, as indicated before, 
the entire process (Steps A-D) is schematic, so that it should eventually be possible to fully automate 
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our method. For this it would be beneficial to have a flexible and accessible conversion of the counter¬ 
models delivered by Nitpick back into Isabelle/HOL input syntax. In fact, an automated conversion of 
Nitpick’s countermodels into the corresponding C*-B and C*-C conjectures would eventually enable a 
truly automated exploration and verification of of the modal logic cube with no or minimal handcoding 
effort. Similarly, for the interactive user a more intuitive presentation of Nitpick’s countermodels would 
be welcome (perhaps similar to the illustrations we used in this paper). 

Using the first-order provers E IfTTl . SPASS |[T9l . Z3 |[T3ll and Vampire |[T^ proved unsuccessful for all 
C*-Isabelle-challenge problems (unless the right lemmas were given to them). Analyzing the reason for 
their weakness, as compared to the better performing higher-order automated theorem provers, remains 
future work. In contrast, the SMT solver CVC4 (via Sledgehammer) was quite successful and contributed 
five C*-Isabelle-challenge proofs. 

Our work motivates further improvements regarding the integration of LEO-II and Satallax: While these 
systems are able to prove all *-Isabelle-challenge problems their proofs cannot yet be easily replayed 
or integrated in Isabelle/HOE. There have been recent improvements regarding the transformation of 
proofs from EEO-II and Satallax to Isabelle/HOE lITSl . using which all the proofs produced by Satallax 
and EEO-II in our work could be checked in Isabelle/HOEQbut this process still requires some manual 
work to adapt the output from the ATPs. 

Our work also motivates further improvements in higher-order automated theorem provers. Eor example, 
for these systems it should be possible to also prove the remaining two *-ATP-challenge problems. More¬ 
over, they needed more than 10 seconds of CPU time in our experiments for the *-Isabelle-challenge 
problems; it should be possible to prove these theorems much faster. 


5 Conclusion 

We have fully verified the modal logic cube in Isabelle/HOE. Our solution is simple, elegant, easy to fol¬ 
low, effective and efficient. Proof exchange between systems played a crucial role in our experiments. In 
particular, we have exploited and combined Nitpick’s countermodel-finding capabilities with subsequent 
calls to the higher-order theorem provers EEO-II and Satallax and the SMT solver CVC4 via Isabelle’s 
Sledgehammer tool. Our experiments also point to several improvement opportunities for Isabelle and 
the higher-order reasoners, in particular, regarding interaction and proof exchange. 

Related experiments have been carried out earlier in collaboration with Geoff Sutcliffe. Similar to and 
improving on the work reported in lO, these unpublished experiments used the TPTP THE infrastructure 
directly. However, in that work we did not achieve a ‘trusted verification’ in the sense of the work 
presented in this paper. Another improvement in this article has been the use of schematic meta-level 
working steps (Steps A-D) to systematically convert (counter)models found by Nitpick into conjectures 
to be investigated. 

Euture work will explore and evaluate similar logic relationships for other non-classical logics, for ex¬ 
ample, conditional logics. Any improvements in the mentioned systems, as motivated above, would be 
very beneficial towards this planned work. Moreover, it would be useful to fully automate the schematic, 
meta-level working steps (Steps A-D) as applied in our experiments. This would produce a system that 
would explore logic relations truly automatically (for example, in conditional logics), analogous to what 
has been achieved here for the modal logic cube. 


*The proofs and the evaluation workflow can be downloaded from http://christoph-benzmueller.de/papers/pxtp2015-eval.zip 
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